Spectral projected gradient and variable metric methods for optimization with linear inequalities

Spectral projected gradient and variable metric methods for optimization with linear inequalities

A family of variable metric methods for convex constrained optimization was introduced recently by Birgin, Mart́ınez and Raydan. One of the members of this family is the Inexact Spectral Projected Gradient (ISPG) method for minimization with convex constraints. At each iteration of these methods a strictly convex quadratic function with convex constraints must be (inexactly) minimized. In the ca...

Spectral Projected Gradient Methods

The poor practical behavior of (1)-(2) has been known for many years. If the level sets of f resemble long valleys, the sequence {xk} displays a typical zig-zagging trajectory and the speed of convergence is very slow. In the simplest case, in which f is a strictly convex quadratic, the method converges to the solution with a Q-linear rate of convergence whose factor tends to 1 when the conditi...

Spectral projected gradient method for stochastic optimization

We consider the Spectral Projected Gradient method for solving constrained optimization porblems with the objective function in the form of mathematical expectation. It is assumed that the feasible set is convex, closed and easy to project on. The objective function is approximated by a sequence of Sample Average Approximation functions with different sample sizes. The sample size update is bas...

Spectral Projected Gradient methods: Review and Perspectives

Over the last two decades, it has been observed that using the gradient vector as a search direction in large-scale optimization may lead to efficient algorithms. The effectiveness relies on choosing the step lengths according to novel ideas that are related to the spectrum of the underlying local Hessian rather than related to the standard decrease in the objective function. A review of these ...

Variable Metric Conjugate Gradient Methods